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ABSTRACT 

Several  recent  studies  of  Naval  Communications  have 
determined  that  the  system  is  not  operating  properly.   This 
paper  presents  a  test  procedure  which  will   improve 
the  performance  of  the  equipment  that   composes  the 
Communications  System.   The  need  for  a  test  procedure,  modes 
of  failure  and  present  test  procedures  are  examined.   It  is 
determined  that  present  test  procedures  referred  to  as  system 
level  tests  are  mislabelled  and  are  actually  subsystems  level 
tests.   A  Systems  Level  Test,  the  Standard  Measurement  Tech- 
nique (SMT)  which  applies  inputs  of  known  degradation  to  a 
system  and  quantifies  the  output  is  presented.   Present 
problems  and  potential  applications  of  SMT  are  discussed. 
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I.   COMMUNICATIONS  AVAILABILITY  AND  THE  NEED  FOR  A  SYSTEM 
TEST  PROCEDURE 

In  recent  years  several  studies  of  the  Naval  Communica- 
tions System  have  been  conducted  such  as  the  Base  Line  II, 

(1),  1966,  Rope  Eval  exercise,  1971,  (2)  Mollohan  report, 
19  72,  (3)  and  the  CNO ' s  Industrial  Advisory  Committee  on 
Telecommunications  (CIACT),  1972,  (4).   It  is  not  the  intent 
of  this  paper  to  restate  the  studies  which  have  been  made  but 
rather  to  show  how  these  studies  support  the  need  of  a  com- 
munications system  test  procedure   which  will  perform  better 
than  our  present  subsystem  level  tests. 

Many  of  the  deficiencies  uncovered  during  one  study 

(which  is  classified  and  cannot  be  identified  in  order  to  keep 
this  paper  unclassified) ,  were  attributable  to  the  fact  that 
an  adequate  method  of  determining  and  monitoring  a  Communica- 
tion System's  Performance  was  not  available.   The  inability 
to  locate  the  cause  of  the  failure  or  degradation  would  very 
often  result  in  placing  the  blame  on  the  other  station  (s)  or 
on  the  atmosphere.   An  element  of  Black  Magic  was  introduced 
because  neither  end  of  the  trunk  knew  where  to  assign  the 
blame,  (itself,  the  other  station  or  the  atmosphere).   There 
was  no  method  available  to  determine  the  system's  level  of 
performance.   There  are  standards  available  by  which  to 
measure  component,  equipment  and  subsystem  performance  but 
there  are  no  standards  to  measure  the  overall  performance  of 
a  system  either  shipboard  or  shore  station.   The  Report  said 


there  is  a  need  to  prepare  and  publish  detailed  standard 
procedures  for  operations ,  maintenance  and  management  of 
Naval  Communications  on  a  systems  level.   The  Report  also 
pointed  out  that  the  "systems  approach"  is  not  used  or  taught 
at  any  level  of  the  Naval  training  establishment. 

The  studies  mentioned  have  attempted  to  measure  the  Naval 
Communications  System's  ability  to  communicate  and  the  con- 
clusions are  that  the  system  is  a  very  poor  performer.   It 
takes  too  much  time,  uses  too  many  assets,  looses  messages 
and,  occasionally,  ceases  to  function. 

The  CIACT  report  called  upon  the  Navy  to  develop  an  over- 
all plan  to  include  a  general  performance  description  for  the 
total  Naval  Communications  system  [4,  p.  17].   It  can  easily 
be  argued,  that  some  means  must  be  available  to  measure  the 
"general  system  performance"  or  the  "description"  is  of 
little  value.   This  implies  the  need  for  a  system  test  pro- 
cedure.  The  CIACT  goes  on  to  say  the  Naval  Communications 
Command  is  not  allowed  billets  for  communications/electronics 
engineers  to  operate,  service  and  manage  a  complex  communica- 
tions system.   With  no  person  ready  or  trained  to  manage  a 
communications  system,  systeir,   accountability  was  not  called 
for,  therefore,  there  was  no  test  of  the  system  in  order  to 
measure  it. 

Now  that  official  attention  has  been  drawn  to  the  need  of 
a  system  test  procedure  to  measure  the  level  of  performance 
of  the  system,  an  attempt  will  be  made  to  specify  what  the 
system  test  procedure  should  accomplish. 


The  test  procedure  should  be  able  to  determine  if  the 
total  system  is  operating  and  if  so,  at  what  level.   No 
system  will  operate  continuously  at  100%  and  if  System  A  is 
operating  in  a  degraded  mode,  will  it  be  able  to  communicate 
with  System  B  which  is  also  operating  in  a  degraded  mode? 
How  much  degradation  can  the  entire  system  accept  and  still 
perform  satisfactorily?   The  test  procedure  must  be  able  to 
determine  how  much  degradation  (either  in  the  input  signal, 
or  within  itself) ,  a  system  can  tolerate.   The  test  procedure 
should  be  able  to  predict  the  system  availability  so  that  the 
user  can  plan  the  execution  of  this  assigned  mission. 

NASA  has  established  long-term  average  system  performance 
goals  for  the  various  transmission  media  (microwave,  land 
line,  submarine  cable,  communications  satellite,  and  HF  radio). 

NASA  circuit  availability  objectives  expressed  on  percent- 
ages ,  indicating  the  portion  of  the  total  scheduled  operating 
time  during  which  the  circuit  is  expected  to  be  available. 

Microwave  and  landlines      99.5% 

Submarine  cables  99.5% 

Communications  Satellite   99.8% 
High-frequency  radio       95.0% 

Table  1-1  NASA  circuit  availability;  adapted  from 
Stelter  [5,  p.  274] 

These  goals,  in  principle  what  the  CIACT  report  calls  for, 
are  expressed  as  percentages ,  indicating  the  portion  of  the 
total  scheduled  operating  time  during  which  the  circuit  is  ex- 
pected to  be  available  to  the  user.   "Circuit"  as  used  here. 


includes  the  medium  of  transmission  and  the  equipment  at 
both  ends;  a  trunk.   NASA  measures  its  circuit  performance 
by  subjecting  it  to  a  bit  error  rate  test  of  24  hours  duration 
on  a  periodic  schedule.   A  Data  Transmission  Test  Set  gener- 
ates a  20  4  7  bit  pseudo  random  pattern  that  is  transmitted  via 
the  modems  normally  associated  with  a  particular  circuit,  to 
a  compatible  receiver.   The  receiver  is  attached  to  a  Print- 
Punch  recorder.   This  recorder  provides  a  time-tagged,  perma- 
nent record  of  the  circuit  measured  bit  error  rate.   In  this 
manner  a  circuit  level  of  performance  is  established  [5,  p. 
272]  . 

Compiling  circuit  performance  levels  enables  NASA  to 
establish  a  system  level  of  performance. 

A  Naval  Underwater  System  Center  study  has  proposed  a 
similar  test  procedure.   The  test  is  made  by  applying  an  in- 
put sequence  X  such  that  the  resulting  output  sequence  will 
be  Z  if,  and  only  if,  the  circuit  is  operating  correctly. 
In  this  manner  a  detailed  knowledge  of  the  logic  elements , 
the  mechanics,  the  theory  and  their  interaction  used  to  im- 
plement the  system  is  not  necessary  in  order  to  test  it.   The 
test  will  detect  almost  all  malfunctions  resulting  from 
single,  simultaneous  or  unanticipated  failures.   [6,  p.  4]. 

The  final  report  on  Evaluation  of  the  unencrypted  Data 
Link  System  of  the  P-3C  airplane  (INSERV  REPORT)  brings  out 
the  need  for  a  system  test  procedure  which  can  isolate  fault 
to  a  component.   It  criticizes  the  AN/ACQ-5  built-in  test 
equipment  which  isolates  faults  to  a  group  of  modules  or  sub- 
systems which  can  contain  up  to  46  components.   [7,  p.  23] 
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The  test  procedure  should  be  able  to  locate  the  cause  of 
the  degradation  or  failure  down  to  the  component  or  piece  of 
equipment  level  in  order  to  ensure  maintainability  and  high 
system  availability. 

In  summation,  the  Communications  System  test  procedure 
should  be  able  to  accomplish  the  following: 

1)  Detect  any  significant  malfunction. 

2)  Quantify  the  level  of  performance  or  determine  the 
level  of  degration  at  which  the  system  is  operating. 

3)  Determine  how  much  degradation  in  the  input  signal  a 
system  can  tolerate  and  still  perform  satisfactorily. 

4)  Locate  the  cause  of  the  malfunction  down  to  the 
component  piece  of  equipment  level. 

5)  Aid  in  improving  system  maintainability. 

A  test  procedure  which  attempts  to  satisfy  these  criteria 
is  presented  in  this  paper.   Following  chapters  will  attempt 
to  show  why  a  test  procedure  is  needed,  what  a  test  should 
measure  and  how  various  present  test  procedures  compare. 
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II.   A  STUDY  OF  AVAILABILITY 

The  ultimate  goal  of  any  system  or  equipment  is  to  fulfill 
the  particular  mission  for  which  it  was  designed.   Before  any 
system  or  equipment  can  fulfill  its  mission,  it  must  be  avail- 
able.  Increased  equipment  complexity,  new  performance  re- 
quirements, and  extreme  environments  have  resulted  in  higher 
failure  rates,  greater  requirements  for  maintenance  and  lower 
availability  of  current  systems.   [8,  p.  117].   There  has  been 
a  steady  improvement  in  component  reliability,  but  at  a  slower 
rate  than  the  increase  in  complexity.   So  systems  and  equip- 
ment reliability  has  fallen  to  the  point  where  the  system 
often  is  not  available  to  the  user  when  and  where  it  is  re- 
quired.  Furthermore,  the  systems  and  equipment  are  so  ex- 
pensive that  it  is  no  longer  practical  to  get  around  this 
problem  by  buying  more  systems  than  are  needed.   The  operators 
have  come  to  realize  that  the  systems  they  have,  must  be  avail- 
able if  they  are  to  fulfill  their  particular  mission.   [9,  p. 
12-1]. 

In  the  50' s  and  early  60' s  system  availability  was  synon- 
ymous with  reliability.   The  prevailing  attitude  was  that  if 
enough  reliability  could  be  built  into  a  system  its  correspond- 
ing availability  would  ensure  system  effectiveness.   This 
attitude  was  fostered  by  NASA  because  it  determined  that  in 
space  work  there  would  be  no  means  to  replace  or  maintain  any 
system  which  had  failed.   NASA  utilized  redundancy  of  critical 
components  to  attain  the  needed  level  of  reliability. 
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Therefore  reliability  was  equal  to  availability  and  all  of 
the  literature  of  the  period  stresses  this  point.   As  systems 
engineers  have  learned  to  understand  and  work  with  reliability 
they  have  realized  that  the  initial  premise  is  true  but  that 
systems  availability  could  be  obtained  by  means  other  than 
reliability  such  as  maintainability. 

It  must  be  recognized  that  systems  availability  not  reli- 
ability is  the  ultimate  criteria  and  that  it  can  be  obtained 
through  reliability  and  maintainability.   This  criteria  can 
be  expressed  mathematically  as  the  relationship  between 
availability  A,  reliability;  mean  time  between  failure  (MTBF) 
and  maintainability;  mean  time  to  repair  (MTTR).   [10,  p.  8-11] 

A  =       MTBF 


MTBF  +  MTTR 


This  paper  supports  that  premise  and  will  attempt  to  show  that 
adequate  testing  is  needed  to  support  maintainability  in  order 
to  achieve  availability.   (Fig.  2-1) 

4 

A.   RELIABILITY 

Today,  although  in  an  environment  of  declining  "technology 
spending",  especially  related  to  defense  and  aerospace,  empha- 
sis is  focused  on  operational  and  system  effectiveness .  Aus- 
terity in  spending,  coupled  with  the  need  for  more  sophisticated 
equipment,  focuses  an  even  more  important  light  on  the  need 
for  a  new  and  more  practical  reliability  engineering  technology. 
The  continued  criticality  of  effective  equipment  performance  in 
sophisticated  systems  creates  a  changing  emphasis  on  reliability 
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and  requires  much  more  careful  control  of  the  design  process. 

[11,  p.  7-1]. 

Reliability  definition:   1)   The  ability  of  an  item  to 
perform  a  required  function  under  stated  conditions  for  a 
stated  period  of  time.   2)   The  characteristics  of  an  item 
expressed  by  the  probability  that  it  will  perform  a  required 
function  under  stated  conditions  for  a  stated  period  of  time 

[12,  p.  198].   This  general  definition  is  just  a  starting 
point;  the  product  specification  for  each  product  should  in- 
clude clear  definitions  of  all  modes  of  operation  important 
to  the  function  of  the  device.   Dormant  and  operating  environ- 
ments as  well  as  life  expectancy  are  necessary  considerations 
in  the  specification. 

A  satisfactory  definition  of  reliability  in  the  product 
specification  should  provide  all  the  information  necessary  to 
judge  whether  device  operation  is  proper  or  improper.   The 
definition  should  state  what  constitutes  failure  and  the  con- 
ditions under  which  failure  occurs.   Reliability  is  usually 
stated  as  the  "mean  time  between  failure"  -  MTBF  along  with  a 
confidence  interval  on  the  mean  time. 

In  order  to  ensure  that  a  system  has  the  required  reliabil- 
ity, DOD  has  found  it  necessary  to  establish  a  formal,  four 
phase,  system  life  cycle.   The  system  consists  of  Concept 
Formulation,  Contract  Definition,  Acquisition  and  Operational 
phases . 

In  the  Concept  Formulation  phase,  the  mission  of  the  system 
is  justified  and  its  long  range  goals  are  stated.   The  mission 
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is  defined  as  reliability  requirements  predictions  are  made 
on  the  basis  of  the  functions  that  must  be  performed  to 
accomplish  system  effectiveness  and  mission  successes.   The 
accuracy  of  these  predicted  requirements  need  only  be  enough 
to  justify  proceeding  with  the  next  phase  of  the  project. 
Alternative  solutions  are  listed  and  feasibility  studies  may 
be  conducted. 

The  Contract  Definition  phase  includes  identification 
and  specification  of  system  characteristics  and  the  prepa- 
ration of  program  plans  which  will  assure  a  high  probability 
of  achieving  the  specified  missions  in  a  cost  effective 
manner.   Modern  program  management  demands  that  specification 


AVAILABILITY 
4 


I 

RELIABILITY 

(Operating  time) 


n 

MAINTAINABILITY 

(down  time) 


TEST  PROCEDURE 


AVAILABILITY 
-  Adapted  from  Blanchard  [8,  p.  18] 

(Fig.  2-1) 

decisions  be  based  on  quantitative  analysis.   Numerical 
values  for  the  system  characteristics,  i.e.,  reliability, 
maintainability,  operability,  safety,  cost  analysis,  quality, 
security,  training,  procedures  and  logistics,  must  be  predicted 
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during  the  Definition  phase  of  the  system  life  cycle.   Reli- 
ability and  maintainability  program  plans  must  be  part  of  a 
total  management  system.   Identification  and  funding  of  reli- 
ability and  maintainability  programs  must  be  an  integral  part 
of  a  program  managers  overall  cost,  schedule  and  technical 
control  system.   [13,  p.  1-5].   These  characteristics  must  be 
repredicted  or  assessed  repeatedly  throughout  the  Acquisition 
phase  and  they  must  be  measured  or  demonstrated  before  and 
during  the  Operational  phase.   These  numerical  values  have 
value  only  when  used  to  give  a  predicted,  assessed  or  mea- 
sured value  of  overall  system  effectiveness. 

While  determining  a  quantitative  reliability  specification, 
certain  factors  must  be  kept  in  mind.   It  is  impossible  for 
man  to  build  a  system  with  100%  reliability  because  of  the 
limits  of  knowledge,  physical  laws  and  monetary  constraints. 
There  will  be  a  reliability  decrement  due  to  design  defi- 
ciencies, manufacturing  deficiencies  and  use.   (Fig.  2-2) 
These  losses  will  occur.   In  addition  to  the  physical  environ- 
ment problem  the  effect  of  the  social  environment  containing 
people,  the  man/machine  interface,  and  their  expectations, 
goals,  motivations,  training,  and  other  psychological  and 
physiological  attributes  will  introduce  some  variance  to  the 
stated  reliability.   [15,  p.  255].   The  system  planners  must 
determine  how  much  reliability  loss  can  be  expected  and  how 
much  can  be  tolerated.   There  are  numerous  mathematical 
methods  of  quantifying  the  reliability  necessary  for  a  par- 
ticular system.   Such  as  Monte  Carlo  models  [16,  p.  4)  and 

regression  analysis. 
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An  additional  input  needed  before  a  level  of  reliability 
sought  can  be  selected  is  cost.   It  must  be  recognized  that 
there  is  a  cost  effective  level  of  reliability  and  that  any 
demand  for  greater  reliability  will  be  disproportionate  with 
'  its  cost.   Above  this  point  some  means  to  achieve  increased 
availability  other  than  reliability  should  be  sought.   Trade- 
offs between  cost,  availability,  reliability  and  maintain- 
ability must  be  analyzed  to  determine  the  cost  effective 
blend. 

With  quantitatively  designed  reliability  values  available 
manufacturing  can  be  monitored  to  ensure  attainment  of  the 
needed  reliability  levels.   Emphasis  must  be  placed  on  the 
fact  that  the  reliability  goals  are  determined  prior  to 
manufacturing  and  are  included  in  the  specifications.   In 
this  manner  all  personnel  working  on,  and  the  ultimate  users 
of  the  item  are  aware  of  the  established  reliability  levels 
and  can  work  to  assure  their  attainment.   This  assumes  the 
existence  of  1)  testing  or  verification  procedures  to  deter- 
mine the  actual  level  of  reliability  and  2)  management  control 
procedures  to  handle  any  variance  between  the  desired  and 
actual  levels  of  reliability  attained.   The  management  con- 
trol procedures  should  be  specified  in  a  management  plan. 
While  reliability  considerations  can  be  very  useful  they  are 
not  a  panacea  as  the  following  examples  will  demonstrate: 

1)   When  using  MTBF  as  a  reliability  criteria  how  is  a 
component  which  does  not  fail  but  goes  into  degraded 
operation  handled.   It  is  not  a  black  and  white  but. 
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a  grey  area.   If  specifications  were  written  to  cover 
all  grey  areas  they  would  be  voluminous . 

2)  Determining  reliability  and  cost  estimates  for 
marginal    analysis  is  expensive  and  time  consuming. 

3)  At  present  the  performance  of  some  systems  is  difficult 
to  quantify  and  does  not  lend  itself  to  quantitative 
values  which  can  be  utilized  for  reliability  specifi- 
cations; i.e.,  voice  communications  systems.   The 
object  of  the  system  is  to  pass  intelligible  infor- 
mation.  How  is  "intelligible"  defined?   MIL-STD-188C 
revised  24  Nov.  1969  gives  the  following  performance 
objectives.   "Tactical  transmission  system  volume, 
loop  losses ,  switch  losses  and  inter-office  trunks 
shall  be  engineered  to  provide  the  following  listen- 
ers preference  ratings  in  terms  of  listeners  prefer- 
ences : 

85%  of  the  users  will  rate  the  circuit  "Good". 

10%  of  the  users  will  rate  the  circuit  "Fair". 

5%  of  the  users  will  rate  the  circuit  "Poor  or 
Worse"."   [17,  p.  70] 
Although  these  are  performance  criteria,  they  do  not  satisfy 
the  criteria  of  quantitative,  reliability  specifications. 

B.   MAINTAINABILITY 

Maintainability  definition:   A  time  function,  the  prob- 
ability, understated  conditions,  that  a  maintenance  process 
will  terminate  successfully  within  a  stated  time  measured 
from  initiation  of  the  process . 


The  maintenance  need  can  be  brought  on  by  degraded 
operation,  catastrophic  failure  or  because  of  the  lapse 
of  a  predicted  reliability  time  period.   The  maintenance 
process  can  be  preventative  maintenance  which  considers  time 
as  a  constant,  either  because  regular  periods  of  fixed 
duration  are  provided  for  the  purpose  or  because  the  tasks 
are  so  routine  that  performance  times  exhibit  negligible 
dispersion.   On  the  other  hand  corrective  maintenance  typi- 
cally involves  unforseen  contingencies  such  as  catastrophic 
failure  or  systems  deterioration  while  still  performing  above 

* 

a  "failed"  level  [18,  p.  137]. 

Reliability  is  one  side  of  the  system  "availability" 
coin.   Though  complex  equipment  can  fail,  reliability  has 
attained  a  degree  of  sophistication  that  makes  it  possible  to 
predict  failure  rates  with  a  fair  degree  of  accuracy.   The 
other  side  of  the  coin,  maintainability,  presents  a  problem. 
Even  though  the  need  for  maintainability  has  been  recognized 
in  some  quarters  it  has  not  received  wide  attention  and 
therefore  has  not  reached  the  stage  where  it  can  be  effec- 
tively quantified  with  any  appreciable  degree  of  acceptance. 
Since  most  things  are  fallible,  maintainability,  or  the  need 
for  it,  is  a  fact  of  life.   When  failure  occurs  repair  is 
required  and  the  faster  the  better.   If  either  "ability" 
fails  to  do  its  job,  or  fails  to  support  the  other,  avail- 
ability suffers.   [19,  p.  1-5]. 

A  useful  description  of  availability  is  contained  in 
NAVSHIPS  94324.   The  mathematical  approached  used  is 
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A  =  R  +  Mq  (1-R) 
where  A  is  the  availability,  R  is  the  equipment  reliability 
and  R-.  is  the  operational  maintainability  of  the  equipment. 
[20,  p.  3]  . 

The  initial  Concept  Formulation  phase,  when  a  future 
system  is  being  considered  and  the  alternatives  are  being 
examined,  is  when  maintainability  long  range  goals  should 
first  be  considered.   From  the  point  in  time  when  reliability 
becomes  a  consideration  maintainability  must  also  be  present. 
In  the  Concept  Formulation  phase,  maintainability  functions 
such  as,  logistics,  support  systems,  personnel  and  training 
must  be  present  and  kept  visible  throughout  the  system 
evolution . 

Maintainability  requires  an  integration  of  design  and 
maintenance  engineering  effort  to  provide  controls  which 
assure  that  the  system  being  designed  is  adequately,  expedi- 
tiously, and  economically  supported.   Maintainability  is  a 
design  parameter  which  reflects  support  considerations,  and 
is  prerequisite  to  integrated  logistics  support.   [8,  p.  117]. 

Quantitative  maintainability  values  must  be  determined  by 
considering  trade-off's  with  reliability  and  cost.   Figure 
2-3a  represents  trade-off  alternatives  between  reliability 
and  maintainability  at  a  constant  availability  level. 
Figure   2  -  3b     shows  the  effect  upon  reliability  and  main- 
tainability of  cost  and  Figure  2-3c  depicts  the  trade-offs 
between  availability,  reliability,  maintainability  and  cost. 
During  this  phase  the  system  characteristics  must  be  determined 
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and   used   as    guidance.      The  basic  methods    of   construction 
(extent   of  modularity,    accessibility,    fastenings,    plug-in 
and  wire-in    features,    etc.) ,    the    location    and   circuit   protec- 
tion  of   test   points,    the    level   of   the    lowest    replaceable    unit 
(LRU's) ,    the   extent   to  which    automatic    fault    detection    (AFD) 
and  built-in   test   equipment    (BITE)    will   be   provided,    the    type 
and  extent   of   fault    isolation   and   localization   software 
(programs    for  AFD   and   technical   manuals    and   data   for  main- 
tenance  men) ,    and  the   nature    and   degree    to  which   on-line 
performance   monitoring   and   testing   is    to   be   provided, must    all 
be    specified  during   the    Contract   Definition   phase.        [21,    p.    33] 

Various   models   have   been    developed  based   upon   mathematical 
distribution   models    to    aid   in   quantifying   maintainability 
[22,    p.    501]     [18,    p.    137]     [20,    p.     3]. 

If  we    are   to   profit    from  the    ground  work   performed   during 
the   previous   phases ,    the   Acquisition    and  Operational   phases 
must  be    guided  by   two   paramount   considerations.      First  we 
must   be   prepared   to   perform   the   management    function   of   con- 
trol.     Controlling   is    the   management    function   of   making   sure 
that   plans    succeed.       In   other  words ,    it   is    the   measuring   and 
correcting   of    activities    to   ensure    that   these    activities    are 
contributing   to   the    achievement   of   planned   goals.        [2  3,    p.    50  7] 
During   the    Concept   Formulation    and   Contract   Definition   phases 
standards    for  maintainability  were   established;    during  Acqui- 
sition   and  Operations    maintainability   performance    must  be 
measured   and   deviations    from   the    standards    corrected. 
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Measurement  is  the  second  paramount  consideration.   Since 
we  are  concerned  with  systems  availability  which  we  plan  to 
enhance  through  systems  maintainability,  we  must  have  a  means 
to  test  the  systems  maintainability  to  be  able  to  quantify 
the  level  of  performance. 

Test  procedures  which  test  the  entire  system  and  not 
components  of  it,  are  very  hard  to  find,  and  are  the  subject 
of  this  paper. 
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III.   MODES  OF  FAILURE 

This  chapter  will  attempt  to  analyze  failure;  its  causes, 
types,  occurrence  and  effects. 

A.   CLASSIFICATIONS  OF  FAILURE 

Failures  can  be  classified  according  to  location  or  cause. 
A  classification  of  defects  according  to  the  location  of 
failure  determines  a  weak  point  in  a  system  and  is  the  first 
step  needed  to  strengthen  it.   The  reason  for  the  occurrence 
of  the  failure  can  be  analyzed  as  follows : 

Design  defects:   Defects  attributable  to  design  imper- 
fections.  Pushing  the  state  of  the  art  or  failure  to  consider 
all  possible  stimuli  and  responses  of  a  system  lead  to  design 
defects  . 

Manufacturing  defects:   Failures  of  this  class  occur  as 
a  consequence  of  violation  of  the  technological  manufacturing 
procedure  chosen  for  the  system  or  unit.   The  quality  of  the 
individual  units  and  components  of  a  system  have  unavoidable 
random  variations.   These  manufacturing  variations,  decrease 
the  reliability  of  some  of  the  items  or  units. 

Improper  use:   For  every  system,  restrictions  are  made 
on  the  conditions  of  its  use.   Rules  are  given  for  maintenance 
and  operation  of  the  system  and  its  parts.   Violations  of  these 
rules  or  restrictions  may  lead  to  premature  failure. 

Deterioration  (aging) :   No  matter  how  high  the  quality  of 
the  unit  and/or  the  system  as  a  whole,  gradual  wear  or  aging 
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is  inevitable.   During  the  course  of  use  and  storage,  ir- 
reversible changes  take  place  in  metals,  plastics  and  other 
basic  materials.   The  cumulative  effect  of  these  changes 
deteriorates  the  strength,  coordination  and  interaction  of 
the  parts.   This  deterioration  may  not  be  as  readily  apparent 
as  an  outright  failure.   The  piece  of  equipment  or  system  is 
still  operating  but  at  a  sub-optimal  level  of  performance. 
All  components,  subsystems  and  systems  are  subject  to  dete- 
rioration which  in  the  final  analysis  may  cause  failure. 

B.   TYPES  OF  FAILURE 

Instantaneous  failure:   Failures  occur  at  the  instant  an 
operational  parameter  is  exceeded.   These  failures  are  random. 
Typically  the  failure  of  the  unit  or  system  occurs  independent- 
ly of  how  long  it  has  been  used  or  what  condition  it  is  in. 
An  example  of  this,  is  connecting  a  110  volt  appliance  to  a 
440  volt  power  supply.   The  failure  is  instantaneous  and 
usually  readily  apparent,  no  test  may  be  necessary. 

Degradation  failure:   This  failure  occurs  as  the  result 
of  the  gradual  change  in  operating  parameters.   Admissible 
limits  for  the  operating  parameters  of  a  unit  or  system  are 
established  when  it  is  designed.   A  performance  parameter 
operating  outside  these  limits  is  classified  as  a  failure. 
[24,  p.  5].   All  units  or  systems  are  subject  to  wear  or  aging 
and  therefore  their  condition  is  continually  changing  or 
transient;  usually  it  is  deteriorating.   A  minimum  performance 
limit  is  established  and  the  unit  or  system  will  approach  this 
limit  throughout  its  lifetime.   This  state  is  known  as 
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"graceful  degradation."   A  test  procedure  is  needed  to 
determine  how  much  degradation  has  occurred. 

Interaction  failure:   A  situation  in  which  several  causes 
of  failure  act  simultaneously.   It  is  the  most  typical  type 
of  failure  in  practice.   A  vacuum  tube  may  fail  because  it 
was  subjected  to  a  random  "peak"  load,  after  graceful  de- 
gradation had  lowered  its  upper  level  limit.   The  result  is 
instantaneous  failure.   Interaction  failure  can  be  reduced  by 
calibration  which  restores  the  upper  limit  to  its  proper 
level,  if  the  amount  of  degradation  is  known. 

C.   OCCURRENCE  OF  FAILURE 

The  occurrence  of  failure  is  examined  by  analyzing  the 
distribution  of  failures  with  respect  to  time ,  especially  for 
components  utilizing  electronic  technology.   Figure  3-1  shows 
the  classical  failure  rate  for  a  finite  population  of  newly 
manufactured  components  under  a  uniform  operating  stress. 
The  failure  rate  curve  is  referred  to  as  the  bathtub  curve. 
It  is  based  on  the  presumption  that  failures  will  be  fixed 
as  fast  as  they  occur,  to  keep  the  component  operating. 
Virtually  all  components  start  their  life  with  a  very  high 
initial  failure  rate,  largely  because  of  design  and  manufac- 
turing deficiencies  which  economical  quality  control  cannot 
find.   This  period  is  known  as  a  debugging,  or  burn-in  or 
infant  mortality  period.   Following  the  burn-in  period,  the 
component  enters  its  useful  life  period.   Toward  the  end  of 
the  useful  life,  so  many  parts  are  deteriorating  so  fast  that 
maintenance  becomes  uneconomical,  the  failure  rate  begins  to 
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rise   and   the    component   enters    its   v/ear-out   phase   or  end 
of   life    [25,    p.    2]  . 

The    classical    failure    rate   has   been   challenged  with    respect 
to   its    validity    from  the   user's    point   of   view    as    applied   to 
electrical    components.      Most   contracts    call    for   a  manufactur- 
er's  burn-in   period  prior   to    acceptance.       For   the   user   the 
burn-in   period   is   nearly   non-existent.      With   integrated 
circuits,    printed   circuits,    chips,    transistors,    etc.,    most 
electrical    components    are   obsolete    and   are    replaced   before 
they   enter   the   wear-out   phase    so   it   does    not    apply. 

Examination   of   the    useful    life   period   calls    for    following 
a   subtle    line   of   reasoning   and   the    solving   of    a   contradiction, 
which   this   paper  will   not    attempt.      The    first    argument   says 
that,    given    adequate   preventative   maintenance,    experience 
has    shown   that   the    failure    rate    is    fairly    constant    during 
this   period.      The    second   argument   says    that   since    the    failure 
rate    during   the    useful    life   period   is    determined  by    random 
failures   preventative    maintenance   will   have   no   effect    upon 
the    failure    rate    and  therefore   serves    no   purpose.      Both 
arguments    accept   useful    life    failure   and   the   existence    of 
degradation  which   may    impede    a   component's    performance   while 
not    lowering   it   to    a    "failure"    status. 

F.       EFFECTS    OF    FAILURE 

The   effects    of    failure    can   be    as    varied   as    the    user,    the 
component   or   the    system  which   has    failed.      The   effect   could 
range    from  insignificance    to    catastrophy    and    loss    of    life. 
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In  order  to  avoid  the  effects  of  failure,  missions  are 
curtailed  or  altered,  duplicate  systems  (redundancy)  are 
employed  and  several  more  expensive  solutions  are  implemented. 
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Figure  3-1 
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IV.   PRESENT  TEST  PROCEDURES 

Before  examining  test  procedures,  equipment  levels  will 
be  defined. 

Component:   A  component,  (subassembly,  assembly,  unit  or 
group)  is  any  collection  of  pieces  which  composes  a  sub- 
division of  a  set,  piece  of  equipment  or  system.   By  itself 
it  is  capable  of  independent  operation  but  it  is  not  capable 
of  performing  a  complete  operational  function.   It  is  usually 
replaceable  as  a  whole,  but  has  parts  which  may  be  individu- 
ally replaceable.   (Examples:   Electronic  power  supply,  radio 
receiver,  antenna  group,  oscillator) . 

Equipment  (set) :   A  component  or  components  and  necessary 
parts  connected  or  associated  together  to  perform  an  opera- 
tional function.   (Examples:   Radio  receiving  set,  sound 
measuring  set,  which  includes  such  parts  and  components  as 
cable,  microphone  and  measuring  instruments;  radar  homing 
set)  . 

Subsystem:  A  subsystem  is  defined  as  a  combination  of 
equipments,  components,  etc.,  which  perform  an  operational 
function  within  a  system. 

Subsystems  from  the  major  subdivisions  of  systems. 
Examples:   (a  radar  station,  fire  control  subsystem,  a  radio 
transmitting  facility. 

System  (Electrical-Electronics) :   A  combination  of  equip- 
ment and/or  subassemblies,  generally  physically  separated 
when  in  operation  and  other  such  components  necessary  to 
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perform  an  operational  function  or  functions.   Examples: 
(Antiaircraft  defense  system  including  tracking  radar,  computer 
and  gun  mount;  GCA  electronic  system;  communications  system: 
A  link  including  transmitter,  medium  of  transmission  and 
receiver).   [26,  p.  4]. 

In  order  to  determine  the  level  of  operation  of  a  test 
procedure  we  must  specify  the  level  of  operation  of  the  item 
being  tested.   The  composition  and  the  characteristics  which 
must  be  measured  at  each  level  must  be  specified.   Once  the 
characteristics  being  tested  have  been  defined,  the  level  of 
operation  of  the  item  and  the  test  system  can  be  defined. 
Table  (4-1)  attempts  to  categorize  and  arrange  the  equipment 
levels  and  the  characteristics  measured. 

A  capsule  summary  of  the  prominent  test  procedures  used 
in  Naval  Communications  will  follow.   It  must  be  understood 
that  variations  in  tests  occur  from  ship  to  ship  and  fleet  to 
fleet,  however  an  attempt  will  be  made  to  provide  a  basic, 
objective  guideline  and  discussion  of  each  procedure. 

The  Quality  Monitoring  System  (QMS)  is  a  subsystem  level 
performance  monitoring  procedure.   It  is  composed  of  hard 
wired,  monitoring  equipment  which  is  interfaced  into  the 
communications  system  at  key  points.   It  can  be  used  on-line 
or  off-line  to  measure  degradation  of  selected  character- 
istics.  It  requires  extensive  training  and  is  expensive  to 
install,  however,  quantum  improvements  in  communications 
system  performance  can  be  obtained  when  QMS  is  properly  used. 
QMS  is  capable  of  performing  qualitative,  voice,  systems 

level  testing.  [27] 
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Pomsee   is    a   subsystem  level    calibration   test.       It   can  be 
performed   aboard   ship   but   its   best    application   is    ashore    at 
a   depot    level   maintenance    facility.      Equipment   must  be    removed 
and   dismantled   and   is    tested   on    a   test  bench.      It   is    costly, 
time    consuming   and   requires    a  high    level   of   training   to   be 
properly   performed.       It   is   best    applied   to   pieces    of   equipment 
which    are   known   to  have    failed   and  not   as    preventative   main- 
tenance  or   in   order   to   locate    a    failure. 

Built-in   test   equipment    (BITE)    is    a   confusing   title    to 
apply   to    a  test  procedure.      QMS    could  be    considered  built-in 
test   equipment,    however    the   term   usually    applies    to    component 
or  equipment    level    "self-test"    procedures.      BITE    is    capable 
of  being    applied   at   the    subsystem  or   system   level,    however, 
it   is   not   done    in   practice   because   of   cost.       It    usually   pro- 
vides   a   GO /NO-GO   indication,    a   continuity    check   of   cables 
and   connections    and   requires    an   insignificant    amount   of   time 
to  perform.      No   special    training   is    required   to    use    the   test. 
Its   primary   draw-backs    are    cost,    one    test    circuit   or   piece 
of   test   equipment   is    required    for  each    item  being   tested, 
and   the    sparcity   of   trouble    shooting   information    available, 
it    usually  won't   say   what    component    failed   or  why.     [2  8,    p. 
4-1] 

The    Standard  Measurement   Technique    (SMT)    is    a  quantitative, 
system   level,    test  which   measures    the    flow   of   information 
being  processed  by    the    system.       It    can   be   hard  wired  or 
patched   in.       It    is    an    inplace,    off-line    test   which    is    capable 
of   quantifying   the    system   ability    to   process    degraded   inputs. 
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It  requires  very  little  training,  is  moderately  expensive 
and  can  be  performed  in  minutes.   SMT  is  being  developed  by 
the  Naval  Electronic  System  Command  and  a  complete  description 
of  the  system  is  provided  in  the  following  chapter. 

The  Planned  Maintenance  System  1PMS)  is  a  subsystem  level, 
off-line  GO /NO-GO  test.   It  can  be  used  on  a  daily  basis  and 
requires  B-school  training  to  perform.   Portions  of  it  can  be 
performed  with  less  training  and  portions  will  require  higher 
training,  more  time  and  a  test  bench.   PMS  can  be  hard  wired 
into  the  system  but  it  is  usually  patched  in,  using  portable 
equipment.   The  use  of  portable  test  equipment  presents  a 
problem  because  the  equipment  is  often  "banged"  out  of  cali- 
bration rendering  it  unreliable.   PMS  is  used  to  locate  a 
failed  piece  of  equipment.   The  3M  cards  are  used  with  PMS. 
The  3M  test  procedure  is  a  subsystem  level  maintenance  pro- 
cedure.  Basically  it  is  a  check  list,  set  of  instructions 
listed  on  a  card  which  describes  a  test,  lists  the  equipment 
necessary  and  gives  step  by  step  instructions  on  how  to  perform 
the  test.   It  can  be  performed  off-line,  or  on  a  test  bench. 
Fleet  studies  have  indicated  that  component  and  equipment  level 
3M/PMS  testing  is  performed  in  the  fleet  but  that  subsystem 
level  testing  is  not  performed.  [29] 

Table  4-2  presents  the  various  test  procedures  and  attempts 
to  show  the  significant  criteria  which  can  be  used  to  compare 
them  with  each  other. 

The  definition  of  each  criteria  used  in  Table  4-2  is  as 
follows : 
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Component,  equipment,  subsystem  and  system  -  as  presented 
at  the  beginning  of  this  section  or  Table  4-1. 

Detect  failure  -  can  the  test  procedure  detect  a  failure 
at  the  highest  level  of  performance  given? 

Detect  degradation  -  can  the  test  procedure  detect  de- 
gradation at  the  highest  level  of  performance  given? 

Quantitative  -  does  the  test  procedure  provide  a  quanti- 
tative read-out  (as  opposed  to  a  GO/NO-GO  type  output)  at  the 
highest  level  of  performance  given? 

Measure  flow  of  information  -  does  the  test  procedure 
have  the  ability  to  measure  the  flow  of  information  in  the 
system? 

Performed  daily  -  is  the  time  needed  to  perform  the  test 
such  that  it  can  be  performed  daily? 

Remove  and  replace  -  does  the  item  to  be  tested  have  to 
be  removed  in  order  to  be  tested? 

High  training  -  is  a  high  level  of  training  (B-school) 
necessary  in  order  to  be  able  to  perform  the  test? 

It  must  be  pointed  out  that  table  (4-2)  does  not  consider 
the  cost  of  each  test  procedure.   Cost  analysis  must  be  con- 
sidered before  any  test  can  be  considered  optimal. 

The  items  which  should  be  given  major  consideration  are 
System,  measure  flow  of  information,  performed  daily  and  high 
level  of  training.   These  are  the  items  which  the  user  of  the 
test  equipment  is  interested  in  and  upon  which  his  judgment 
will  decide  if  he  will  use  the  test  procedure. 
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V.   STANDARD  MEASUREMENT  TECHNIQUE 

A.   GENERAL  CONCEPT  OF  STANDARD  MEASUREMENT  TECHNIQUE 

Studies  and  communication  exercises  performed  over  the 
years  have  pointed  out  again  and  again,  the  fact  that  an 
adequate  method  of  determining  communications  SYSTEM  perfor- 
mance does  not  exist.   This  chapter  will  attempt  to  offer  a 
solution  to  this  problem. 

Performance  evaluation  has  been  confined  to  a  lengthy 
and  complicated  (POMSEE  etc.)  evaluation  of  individual  equip- 
ments and  proceeded  on  an  assumption  that  proper  operation  of 
system  components  ensured  satisfactory  operation  of  the  total 
system.   This  viewpoint  ignores  the  fact  that  systems  are 
often  synergistic  and  presupposes  component  performance  levels 
are  linear  and  arithmetically  additive. 

The  emphasis  on  individual  equipment  performance  often 
resulted  in  catastrophic  equipment  failure  as  the  sole  criteria 
for  maintenance  action  while  quantum  system  performance  de- 
gradations went  largely  unnoticed.   [30,  p.  3  -  1  to  3  -  17] . 

A  major  weakness  of  the  conventional  methods  for  testing 
and  maintenance  of  communications  systems  is  the  inability  to 
test  for  varying  levels  of  degradation  in  the  system.   Attempts 
to  establish  a  communications  circuit  using  ostensibly  opera- 
tional equipments  often  resulted  in  failure.   Diagnosis  of 
circuit  malfunctions  did  not  isolate  faults  to  the  ship  or 
shore  termination,  the  atmosphere,  or  the  identification  of 
faulty  system  components  at  the  specified  site. 
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The  genesis  of  some  of  these  problems  can  be  found  early 
in  the  equipment  life  cycle.   During  acquisition  equipments 
are  tested  by  various  parameters,  levels,  procedures  and 
conditions.   The  ambiguities  in  equipment  performance  levels 
often  limit  the  amount  of  predictive  analysis  a  communications 
systems  engineer  can  perform  prior  to  system  implementation 
resulting  in  performance  requirements  written  post  facto. 

Subsequent  system  testing  (OPCAP,  INSURV,  etc.)  frequently 
utilize  a  shore  termination  as  an  operational  standard.   Since 
the  standard  has  not  been  tied  to  a  reference  point  utilizing 
strict  methodology  the  predictive  performance  aspects  of  the 
system  operating  with  different  terminations  often  suffer. 

A  need  for  formalized  testing  methodology,  from  equipment 
level  to  systems  level,  is  evident.   An  additional  necessity 
is  the  establishment  of  minimum  system  performance  levels. 
All  too  often  system  levels  are  established  at  or  near  design 
levels  (as  new  or  refurbished  equipments)  without  an  adequate 
determination  of  allowable  performance  deterioration  due  to 
aging.   As  a  consequence,  equipment  replacement  or  overhaul 
can  become  a  qualitative  judgment  without  adequate  consider- 
ation of  an  equipments'  effect  on  its  suit  or  ancillary 
equipment. 

The  Standard  Measurement  Techniques  (SMT)  will  fill 
this  long-standing  need.   The  SMT  will  provide  ships  and  shore 
stations  with  techniques  and  procedures  for  testing  and 
measuring  overall  communications  effectiveness  on  a  system, 
subsystem,  equipment,  and  circuit  level  without  an  emphasis 
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on  automation,  the  need  for  general  purpose  digital  com- 
puters, or  highly  trained  operators  and  technicians.   With 
the  use  of  magnetic  tape  inputs  and  the  traditional  approach 
to  communication  testing,  a  valuable  new  technique  is  developed, 

The  evaluation  of  system  operation  can  be  determined  rap- 
idly by  the  following  technique:   Prerecorded  teletype  signals 
with  various  levels  of  degradation,  such  as  distortion  and 
atmospheric  disturbance  will  be  available  on  magnetic  tapes. 
The  tapes  allow  a  communication  system  to  be  checked  under 
varying  conditions  that  are  experienced' in  actual  ship  and 
shore  operations . 

The  National  Bureau  of  Standards  and  the  Pacific  Missile 
Range  will  pretest,  verify  and  supply  the  magnetic  tapes. 

The  various  test  signals  and  the  utilization  of  existing 
displays  will  permit  interpretation  on,  a  GO /NO-GO  basis,  the 
ability  of  the  system  to  meet  functional  specifications. 

To  provide  for  universal  testing  it  is  necessary  to 
accomplish  the  following  long  range  goals : 

a.  Define  and  describe  present  systems. 

An  Engineering  Data  Register  (EDR)  has  been  promul- 
gated to  define  the  pertinent  characteristics  and 
specified  parameter  values  for  each  Naval  Communica- 
tions System. 

b.  Establish  numerical  systems  performance  levels. 
Reference  Level  Measurement  Guides  (RLMG)  will 
establish  acceptance  levels  criteria  for  systems 
and  equipment.   They  will  specify  the  following 
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for  a  particular  communications  system: 

-  which  parameter  is  to  be  tested. 

-  the  value  or  level  for  measured  parameter. 

-  which  measurement  method  in  the  SMT  should  be  used. 
c.   Develop  techniques  for  measuring  system  performance 

levels.   The  SMT  defines  the  applicable  technique 
for  measuring  each  characteristic  in  conjunction 
with  testing  the  overall  system  performance. 

In  a  letter  to  the  Chief  of  Naval  Operation  (CNO)  ,  dated 
21  Sept.  19  72,  the  Commander,  Naval  Electronic  Systems  Command 
stated  the  following: 

"The  purpose  of  SMT  is  two-fold.   First,  it  measures 
communications  system's  performance  to  determine  system's 
operational  capability.   Secondly,  it  has  the  capability  to 
locate  or  identify  equipments  that  are  not  performing  properly. 
The  tests  are  conducted  on  record,  and  voice  communications 
systems  under  simulated,  controlled,  repeatable  conditions 
that  exercise  all  parameters  of  a  system.   The  test  can  be 
performed  at  sea  or  in  port  without  need  for  assistance  from 
other  ships  or  shore  stations."   [31]. 

In  order  to  contrast  the  SMT  approach  with  present  testing 
methods,  the  following  distinctions  are  made: 

Philosophy  of  present  testing  methods :   The  basic  principle 
of  the  current  testing  methods  is  that  the  better  a  system  is, 
the  less  it  will  distort  the  most  perfect  signal.   Tests  are 
conducted  for  distortion  of  a  single  characteristic. 
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Philosophy  of  SMT  testing  methods:   The  basic  principle 
of  testing  methods  with  SMT  is  that  the  better  a  system  is, 
the  greater  degradation  of  input  it  will  withstand  and  con- 
tinue to  operate.   Tests  are  conducted  for  system  response 
to  maximum  distortions  of  multiple  characteristics.   [32] 

Six    types  of  system  degradation  are  examined  with  SMT: 

Teletype  Distortion:   Teletype  systems  are  tested  by 
introducing  five  lines  of  test  in  successively  increased 
amounts  of  distortion  from  a  tape,  each  line  containing  a 
different  type  of  distortion.   The  terminal  teletype  of  the 
subsystem  is  then  checked  to  determine  at  what  point  the 
system  fails  to  produce  an  error-free  copy. 

Atmospheric  Distortion:   Teletype  systems  are  tested 
by  introducing  ninety  lines  of  test  with  Multipath ,  Noise  and 
Fade  from  a  tape  to  check  the  system's  diversity  capability 
and  the  receiver  subsystem's  response  to  signal  amplitude 
fluctuation  and  sensitivity  (S/N) .   The  terminal  teletype  of 
the  system/subsystem  under  test  is  examined  to  determine  the 
system's  capability  to  produce  an  acceptable  copy. 

Transmitter  Spectral  Purity:   A  system's  transmitter 
is  tested  by  examining  its  output  with  a  spectrum  analyzer 
for  percentage  of  modulation,  harmonic  distortion,  inter- 
modulation  distortion,  and  suppressed  carrier  level. 

Articulation  Index  (AI) :   Voice  systems  are  tested  by 

inputing  various  audio  frequencies  and  a  precise  amount  of 

noise  from  a  tape.   The  receiver  output  is  then  measured 

with  a  noise  measuring  set  to  determine  the  intelligibility 

capability  of  the  system. 
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Multicoupler/Coupler  Swept  Frequency  Response:   A 
system's  multicoupler  is  tested  by  introducing  various 
frequencies  from  a  tracking  generator  into  the  system's  multi- 
coupler  and  examining  its  output  with  a  spectrum  analyzer  to 
determine  loss  or  gain,  bandpass,  ripple  factor  and  skirt 
characteristics . 

Antenna  Swept  Frequency  Response:   A  system's  antenna 
is  tested  by  introducing  a  continuous  spectrum  sweep  from  a 
tracking  generator  at  the  antenna  through  a  directional 
bridge.   The  output  is  then  examined  on  a  spectrum  analyzer 
to  determine  the  standing  wave  ratio  and  resonant  frequency 
of  the  antenna.   The  antenna  signature  presented  on  the 
spectrum  analyzer  is  photographed  and  maintained  as  a  main- 
tenance standard.   [32] 

B.   SPECIFIC  OBJECTIVES  OF  SMT 

We  are  primarily  concerned  with  the  questions  of  whether 
communication  equipment  is  operational,  and  at  what  degradation 
level  it  becomes  inoperable.   Therefore,  the  scope  of  the 
Standard  Measurement  Techniques  program  involves  four  levels 
of  testing,  each  progressively  more  detailed.   Tests  are: 

1.  SYSTEM  TEST 

2.  SUB-SYSTEM  TEST 

3.  EQUIPMENT  TEST 

4 .  CIRCUIT  TEST 

The  tests  are  designed  with  the  following  objectives: 
a.   Test  for  system  response  to  maximum  distortion  of  a 
multiple  of  characteristics  with  a  minimum  of  tests. 
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Reliability;  tests  are  to  duplicate  real  world 
communications  environments  and  will  measure  system 
performance  under  worst-case  conditions.   Systems 
orientation;  emphasis  on  providing  realistic  simulated 
inputs  to  systems  and  observation  of  resultant  outputs. 
Define  numerical  system  parameters  for  inclusion  in  a 
Re  f  e  re  nee  Leye  1  Measurement  Guide  (RLMG)  and  establish 
acceptance/rejection  criteria.   Performance  level 
indices;  numerical  analysis  of  system  performance  vice 
a  reliance  on  GO /NO  GO  judgments  to  provide  a  measure- 
ment of  system  quality. 

b.  Utilize  the  existing  system  equipments  to  interpret 
results.   Economy;  an  avoidance  of  automation,  com- 
puters, and  highly  trained  personnel.   Design  goal  is 
to  provide  total  SMT  capability  for  less  than  $5000.00 
per  site. 

c.  Minimize  the  number,  sophistication  and  time  of  tests 
and  technicians.   Speed;  complete  systems  tests  in 
less  than  15  minutes  per  test.   Simplicity;  test 
procedures  that  are  straightforward  and  easily  under- 
stood by  operational  personnel  at  the  fleet  level. 

d.  Maximize  confidence  in  the  validity  of  system  acceptance 

e.  Establish  capability  to  rapidly  and  confidently  as- 
certain particular  system,  subsystem  or  equipment 
characteristics  which  do  meet  specified  parameter 
values  or  tolerances. 
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f.   Commonality  for  all  communications  systems.   Inde- 
pendence; performance  of  SMT  measurements  that  are  RF 
silent,  self  contained,  and  can  be  accomplished  with- 
out support  of  off-site  activities .   Devise  and 
establish  Standard  Measurement  Techniques  (SMT)  for 
testing  and  measuring  system  performance  levels  of 
communications  systems  employed  at  communications 
stations  afloat  and  ashore.   Develop  and  prepare 
Detailed  Test  Plans  (DTP)  tailored  to  a  specific 
platform/site  and  establish  procedures  for  reporting 
results  of  testing  performed.   [33,  p.  2] 

C.   TEST  PROCEDURES 

The  RLMG  describes  system  tests  necessary  to  ascertain 
that  equipment  characteristics  meet  the  specifications  listed 
in  the  EDR. 

Test  Notes 

'a.   Each  test  procedure  is  furnished  with  a  block  diagram 
for  use  during  test  set-up.   The  test  equipment  is 
shaded  for  clarity. 

b.  A  one  hour  warm-up  period  is  recommended  to  stabilize 
the  system  and  test  equipment. 

c.  Safety  Procedures:   NAVSHIPS  250-660-42,  Electric 
Shock  Its  Causes  and  Prevention,  should  be  made 
available  to  personnel  engaged  in  testing. 

d.  Equipment  that  is  essentially  passive,  e.g.,  receive 
and  transmit  switchboards  and  interconnecting  lines, 
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is  assumed  to  have  been  tested  prior  to  start  of 
system  tests.   [34,  p.  1-2]. 
Functional  System  Tests  (Figure  5-1)  can  be  used  for 
systems  orientation. 

System  Test  Part  1.   TTY  end  equipment. 

System  Test  Part  2.   System  Test  with  TTY  Distortion 

System  Test  Part  3.   Receive  System  Test  with  Atmospheric 

Distortion 
System  Test  Part  4.   Antenna  (Transmit  and  Receive) 

System  Test  Part  1:   TTY  end  equipment 

System  Test  No.  1  is  a  teletype  "end  equipment"  functional 
check  of  reperforator  and  page  printer  only  on  a  closed  loop 
basis  via  the  DC  patch  panels.   Teletype  distortion  tape: 
Five  lines  of  test,  each  line  containing  a  different  type  of 
distortion,  line  1,  MARK  BIAS,  line  2  SPACE  BIAS,  line  3 
SWITCHED  BIAS,  line  4  SPACE  END,  and  line  5  MARK  END  will  be 
transmitted  in  successively  increased  amounts  of  distortion, 
5%  increments,  through  45%  distortion.   The  end  equipment  or 
teletype  of  the  system/sub-system  under  test  will  be  checked 
to  determine  at  what  point  the  teletype  machine  fails  to 
produce  an  error-free  copy.   [35,  p.  ii] 

Procedure  details  for  system  test  part  1: 

a.  Patch  the  tape  recorder  to  the  DC  patch  panel.   Patch 
DC  patch  panel  to  page  printer. 

b.  Send  type  test  message;  TEST  THE  QUICK  BROWN  FOX  JUMPS 
OVER  THE  LAZY  DOG'S  BACK  1234567890  testing. 

c.  Verify  printed  test  message  at  page  printer. 
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Figure  5-1  Functional  System  Block  Diagram 
Multichannel,  Duplex,  AFTS,  RATT 
"P"  System 
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d.  Repeat  test  message  for  each  channel  of  "P"  system. 

e.  Objective  of  the  test  is  to  ensure  proper  operation 
of  perforators  and  page  printers .   Figure  5-2  is  a  block 
diagram  of  the  test.   Table  5  -  1  is  an  example  of  the 
printed  page  messages. 

System  Test  Part  2:   System  test  with  TTY  distortion 

System  Test  Part  2  is  a  quantitative  semi-full  system 
loop  test.   The  following  technique  will  demonstrate  the  over- 
all system  performance  and  its  capability  to  perform  with 
known  distorted  signals.   Input  to  the  system  is  provided  by 
a  magnetic  tape  recorder.   The  tape  recorder  will  have  four 
(4)  instrumentation  channels.   Three  instrumentations  channels 
will  have  prerecorded  teletype  signals  which  are  degraded  at 
successive  intervals  with  various  levels  and  types  of  dis- 
tortion such  as  space  bias  distortion  and  spacing  end 
distortion.   The  fourth  instrumentation  challel  will  be  used 
for  the  tape  recorder  speed  servo  control  loop.   The  test  tape 
used  in  test  Part  1  is  used  for  this  test.   [33,  p.  7]. 
Procedure  details  for  system  test  Part  2. 

a.   The  tape  recorder  is  patched  via  an  interface  unit, 
to  the  DC  patch  panel.   The  transmit  antenna  lead  is  removed 
from  the  antenna  and  patched  to  a  dummy  load.   The  transmitter 
is  patched  to  the  monitor  equipments .   The  receive  antenna 
lead  is  removed  from  the  antenna  and  patched  to  the  monitor 
equipment.   This  series  of  connections  makes  a  loop  permitting 
a  test  signal  to  be  introduced  by  the  tape  via  the  DC  patch 
panel.   The  test  signal  is  sent  through  the  transmit  system 
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to  the  monitor  equipment,  to  the  receive  system  to  the 
page  printer. 

b.  Send  tape  test  message  TEST  THE  QUICK  BROWN  FOX  JUMPS 
OVER  THE  LAZY  DOG'S  BACK  1234567890  TESTING. 

c.  Verify  printed  test  message  at  the  page  printer. 
When  a  NO-GO  condition  appears  maintenance  work  can  be 
accomplished  by  use  of  the  Sub-System  test  1 ,  after  all 
systems  tests  are  complete. 

d.  Repeat  message  for  each  channel  of  "P"  system  or  by 
using  an  interface  panel  test  all  channels  at  one  time. 

e.  The  objective  of  this  test  is  to  ensure  proper 
operation,  compatibility  and  performance  of  the  system,  ex- 
cluding the  two  antenna  systems ,  the  effects  of  the  propa- 
gation medium  and  the  transmit  TTY  equipment.   A  block  dia- 
gram, Fig.  5-3,  is  supplied  for  guidance,  and  table  5-2 
lists  the  equipment  necessary  to  perform  the  test. 

System  Test  Part  3:   Receive  System  test  with  atmospheric 
distortion. 

System  Test  Part  3  is  a  quantitative,  receiving  system 
test.   The  following  technique  will  directly  imply  the  level 
of  performance  to  which  the  system  will  operate.   Input  to 
the  system  is  provided  by  the  magnetic  tape  recorder  as  used 
in  Test  No.  1.   Three  instrumentation  channels  will  have  pre- 
recorded multichannel  RATT  messages  which  are  degraded  at 
successive  intervals  with  various  levels  and  types  of  atmos- 
pheric conditions  such  as  time  spread  distortion  and  noise. 
The  fourth  instrumentation  channel  will  be  for  the  tape 
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LIST  OF  TEST  EQUIPMENT  FOR  SYSTEM  TEST  PART  2 

[34,  p.  2  -  8] 


Description 

1.  Magnetic  Tape  Deck 

2.  Interface  Unit 

3.  Spectrum  Analyzer 

4.  Frequency  Counter/Generator 

5.  Attenuator  120  db 

6 .  Attenuator  12  db 

7.  Micro  Voltmeter 

8 .  Dummy  Load 

9.  Power   Dividers    (2) 

10 .  Line  Fuse  for  Spectrum  Analyzer 

11.  Test  Tapes 

12.  Patch  Cords 


Table  (5  -  2) 
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recorder  speed  servo  control  loop.   Atmospheric  Distortion 
tape:   Ninety  lines  of  test,  including  thirty  lines  of  MULTI- 
PATH  distortion  to  determine  the  system  diversity  capability. 
Thirty  lines  of  FADE  to  exercise  the  receive  systems  capability 
to  respond  to  signal  amplitude  fluctuations  (AGC) .   Thirty 
lines  of  NOISE,  to  test  the  receive  systems  sensitivity  (S/N) . 
In  all  cases,  the  end  equipment,  teletype,  of  the  system/sub- 
system under  test  will  be  checked  to  determine  the  systems 
capability  to  produce  an  acceptable  copy.   (1  error  in  ten 
lines  acceptable,  for  the  system  under  test.)   [33,  p.  11] 
Procedure  details  for  system  test  part  3: 

a.  A  synthesizer  is  utilized  to  supply  an  RF  input  to  a 
mixer  in  conjunction  with  the  input  from  the  tape  recorder. 
The  test  signal  is  processed  through  the  receive  system  (in- 
put is  below  the  antenna)  to  the  page  printer. 

b.  Send  recorded  RF  signals,  with  atmospheric  distortion. 

c.  Verify  printed  test  message  at  the  page  printer. 
When  a  NO-GO  condition  appears  maintenance  work  can  be 
accomplished  by  use  of  the  Sub-System  2,  after  all  systems 
tests  are  completed. 

d.  Repeat  test  signal  for  each  channel  of  "P"  system  or 

by  using  an  interface  panel  test  all  channels  at  the  same  time. 

e.  The  objective  of  this  test  is  to  demonstrate  the 
operability  and  performance  of  the  receiving  system  when 
exposed  to  the  varying  effects  introduced  by  a  test  signal 
with  known  quantities  of  atmospheric  distortion.   A  block 
diagram,  fig.  5-4,  is  supplied  for  guidance  and  table  5-3  is 
a  list  of  equipment  needed  to  perform  the  test. 
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LIST  OF  TEST  EQUIPMENT  FOR  SYSTEMS  TEST  PART  3 

[34,  p.  2  -  15] 


Description 

1.  Magnetic  Tape  Deck 

2.  Test  Tapes 

3.  Synthesizer 

4.  Mixer 

5.  Attenuator  120  db 

6.  Attenuator  12  db 

7.  Micro  Voltmeter 

8.  Power  Divider 

9 .  Line  Fuse 
10.  Patch  cards 


Table  (5 
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Systems  test  Part  4:   Transmit  and  Receive  Antennas 

System  Test  Part  4  is  a  transmit  and  receive  antenna 
test.   Input  to  the  system  is  provided  by  the  magnetic  tape 
recorder.   An  instrumentation  channel  will  have  a  prerecorded 
two-tone  signal.   This  two-tone  signal  is  supplied  to  the 
transmitter  with  a  full  power  setting  utilized  for  the  trans- 
mit antenna.   The  transmit  and  receive  antenna  signals  are 
evaluated  at  the  Current  Monitoring  Panel  (CMP) .   The  receive 
antenna  signal  is  provided  by  the  transmit  loop  of  the  "P" 
system  at  a  minimum  level  to  prevent  multicoupler  overloading. 
[33,  p.  15] 

Procedure  details  for  system  test  Part  4. 

a.  Patch  tape  recorder  to  the  transmitter  Switchboard. 
Attach  a  current  coil  to  transmitter  antenna  lead 

and  to  the  receive  antenna  lead.   Connect  both  current 
coils  to  a  current  monitor  panel.   Patch  a  spectrum 
analyzer  to  the  current  monitor  panel. 

b.  Record  spectrum  analyzer  display  and  determine  the 
standing  wave  ratio  and  resonant  frequency  of  the 
antenna. 

c.  Send  taped  two-toned  signal. 

d.  Repeat  test  for  each  antenna. 

e.  The  objective  of  this  test  is  to  determine  the 
efficient  and  inefficient  frequencies  (or  frequency 
ranges)  for  each  antenna.   Reference  Fig.  (5-5)  for 
a  block  diagram  and  table  (5-4)  for  a  list  of  equip- 
ment necessary  to  perform  the  test. 
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LIST  OF  TEST  EQUIPMENT  FOR  SYSTEMS  TEST  4 
[34,  p.  2  -  21] 


Description 

1.  Magnetic  tape  deck 

2.  Test  tapes 

3.  Spectrum  Analyzer 

4.  Antenna  current  monitoring  panel 

5.  Current  coils 

6.  Patch  cords 


Tabic  (5  -  4) 
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Sub-System  Tests  (Second  level  of  the  SMT) 

The  Sub-System  Tests  are  designed  to  isolate  a  particular 
part  of  the  communication  system  when  a  NO-GO  condition 
appears  during  System  Tests  No.  2  or  No.  3.   There  are  two 
tests,  two  parts  to  each  test.   By  a  process  of  elimination, 
each  test  will  pinpoint  the  NO-GO  equipment  or  equipments 
for  that  particular  test. 

The  other  components  in  the  circuits  are  essentially 
passive,  i.e.,  receiver  and  transmit  switchboards  and  inter- 
connecting lines.   Existing  standard  tests  can  be  utilized 
for  these  items.   The  foregoing  technique  isolates  the 
transmit  and/or  receive  section  of  the  overall  system  to  en- 
able the  test  personnel  to  pinpoint  the  exact  NO-GO  equipment. 
[33,  p.  18] 
Procedures  for  Sub-System  Test  No.  1 

a.   Set  up  test  and  monitor  equipment  as  in  Systems  test 
Part  2  except  that  the  tape  recorder  is  patched  to 
the  transmitter  switchboard  instead  of  the  D.  C. 
patch  panel.   The  tape  recorder  sends  the  test 
■  signals  through  the  transmit  section  of  the  "P" 
system.   After  the  transmitter  is  tested  and  verified 
the  test  tape  input  reverts  back  to  the  multichannel 
keys  and  the  test  is  run  again.   Transmitter  Spectral 
Purity:   The  transmitter  under  test  will  be  tone 
modulated,  8  or  16  tones,  as  required  and  the  output 
will  be  examined  on  a  spectrum  analyzer  to  determine 
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the  spectral  purity,  (%  of  modulation,  harmonic 
distortion,  intermodulation  distortion,  and  suppressed 
carrier  level)  that  applies  to  the  transmitter  under 
test . 
b.   A  black  diagram,  Fig.  (5-6)  is  provided  and  table 
(5-5)  is  a  list  of  equipment  necessary  to  perform 
the  test. 
Procedures  for  Sub-System  test  2: 

a.  Patch  the  tape  recorder  to  the  receiver  switchboard 
and  test  the  multichannel  convertor.   After  the 
convertor  is  tested  test  the  receiver  as  in  System 
Test  Part  3. 

b.  A  block  diagram,  Fig.  (5-7)  is  provided  and  table 

(5-6)  is  a  list  of  equipment  necessary  to  perform 
the  test. 
Voice  Communications  System  Test 

The  Voice   ystem  Test  is  a  test  procedure  whereby  a 
systems  ability  to  transfer  intelligible  information  is 
measured  by  reference  to  an  articulation  Index  (AI) .   "AI 
has  been  shown  to  be  a  function  of  the  link  description  (type 
and  frequency)  ,  and  the  strength  of  the  derived  and  interfer- 
ing signal,  SRF  and  I,  respectively;  hence,  for  any  given 
communications  systems  of  such  configuration  that  link  and 
interferer  descriptions  and  frequencies  are  known.   AI  can 
be  established  as  a  function  of  SRF  and  I."   [36,  p.  1] . 
Values  of  AI  have  been  determined  in  terms  of  audio  S/N 
values  permitting  voice  system's  intelligibility  to  be 
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Figuro~5-6  Transmitter  Test 
Sub-System  Test  1  "P*  System 
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LIST  OF  TEST  EQUIPMENT  FOR  SUB-SYSTEM  TEST  1 

[31,  p.  3-4] 


Description 

1.  Magnetic  Tape  Deck 

2.  Frequency  Counter/Generator 

3.  Spectrum  Analyzer 

4.  Attenuator  120  db 

5.  Attenuator  12  db 

6.  Micro  voltmeter 

7.  Dummy  Load 

8.  Power   Divider 

9 .  Test   Tapes 
10 .  Patch   Cords 


Table     (5-5) 
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Figure  5-7  -Multichannel  Test 


Sub-System  Test  2  "P"  System 
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LIST  OF  TEST  EQUIPMENT  FOR  SUB-SYSTEM  TEST  2 

[31,  p.  3  -  9] 


Description 

1.  Magnetic  Tape  Deck 

2.  Test  Tapes 


Table  (5  -  6) 
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measured  in  terms  of  a  S/N  ratio.   [37].   Articulation 
Index  (AI) :   A  tape  containing  various  audio  frequencies  and 
a  precise  amount  of  noise  will  be  introduced  into  the  system 
and  the  receiver  output  will  be  measured  with  a  noise 
measuring  set  to  determine  the  intelligibility  capability 
of  the  system.   This  test  is  only  used  in  voice  systems. 
[31 ,  p.  iii] . 
Procedure  details  for  voice  system  test: 

a.  A  taped  signal  containing  audio  frequencies  and 
noise  is  processed  through  the  receiver  and  the 
output  signal  is  measured  by  a  noise  measuring  set. 
The  noise  measuring  set  looks  at  the  noise  passed  by 
3  filters  located  in  the  voice  band  width.   The 
difference  between  the  input  and  output  noise  levels , 
expressed  as  S/N  is  equated  to  AI  which  is  equated  to 
intelligibility. 

b.  This  test  procedure  is  used  to  determine  the  perfor- 
mance of  a  receiver  configured  for  voice  communications 
only. 

c.  Figure  5-8  provides  a  block  diagram  of  the  test  and 
table  5-7  provides  a  list  of  equipment  necessary  to 
perform  the  test. 

D.   APPLICATIONS  AND  RESULTS 

Demonstrations  have  been  conducted  on  board  fleet  units 
and  at  shore  activities  to  validate  design  decisions  made 
during  SMT  development,  examine  the  universality  of  the 
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Figure  5-8 
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VOICE  COMMUNICATIONS  TEST  EQUIPMENT.    [37] 
Description 

Random  Noise  Generator 

True  RMS  Voltmeter 

Amplifier 

Wave  Analyzer 

Voice  Interference  Analysis  System 

Digital  Voltmeter 

Tape  Recorder 

Noise  Measuring  Set 

Synthesizer 

Attenuator 

Attenuator/Rotary 


Table  5 
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procedures,  and  to  introduce  communications  system  users  to 
the  capabilities  of  SMT.   Demonstrations  have  been  conducted 
at  or  on  board  USS  England,  USS  Preble,  USS  Intrepid, 
NAVCOMMSTA  SFran,  Naval  Postgraduate  School,  Monterey;  USS 
Constellation  and  other  Commands . 

SMT  performance  has  demonstrated  that  the  original  Long 
Range  Goals  can  be  satisfied.   A  number  of  present  systems 
have  been  defined  and  described  in  the  EDR.   Numerical  sys- 
tems performance  levels  have  been  established  in  the  RLMG 
and  SMT  has  shown  the  techniques  for  measuring  the  systems 
performance  levels.   Several  specific  SMT  objectives  have 
also  been  satisfied.   SMT  tests  are  straightforward  and  can 
be  understood  by  operating  personnel  at  the  fleet  level 
after  a  short,  formal  course  at  Radiomen  "B"  School.   To  a 
large  extent,  existing  test  equipment  can  be  utilized. 
Seven  tests  permit  the  entire  communications  system  to  be 
examined.   Test  sophistication  is  placed  in  the  test  equip- 
ment, not  in  its  operation  or  in  the  technicians.   SMT  test 
sequence  can  be  applied  to  any  communications  system,  afloat 
or  ashore,  unaided. 

During  December,  1972  the  USS  Constellation  (CVA-64)  was 
used  for  a  demonstration  and  some  of  the  test  results  are 
presented  here.   All  communication  systems  available  for 
operation  were  tested.   Several  significant  problems  un- 
covered were  corrected  or  brought  to  the  operator's  attention 
Noteworthy  are  the  facts  that: 
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1.  The  SMT  disclosed  severe  maladjustments  and  failures 
not  discernible  by  other  test  methods  and 

2.  The  time  taken  to  accomplish  the  tests  was  con- 
servatively less  than  five  percent  of  that  required 
by  more  conventional  means  available  to  the  ship 
personnel . 

Problems  encountered  which  could  be  corrected  by  adjust- 
ments and/or  alignments  were  corrected.   Problems  requiring 
maintenance  or  repair  were  explained  to  cognizant,  responsible 

personnel . 

It   must   be   pointed   out   that    the   test   sequence   permitted 
testing   of   parameters    not   testable   by   other   methods    and   that 
the    results   were    able    to   point   out   pieces    of   equipment  which 
were    functioning  but   operating  out   of   specifications.     [38] 

Table    5-8    is    an   example   of   the    SMT  TTY   test.      After   the 
page   printers   were    checked,    the    alternate    channel    convertors 
for   the    "P"    system    (UCC-1    channels    1    to    16)    were    tested    and 
6    out   of    8  were    found   in   need   of   maintenance.      All    "n" 
system  channels     (low   tones)    were    adjusted   and   their   perfor- 
mance  was    greatly   improved   after   the    SMT   tests    pointed   out 
weaknesses.      A   total   of    23    transmit    and    receive    multicouplers 
were   tested.      Ten   of   the    23   were    found   to   be   out   of   specifi- 
cations.     Twenty-one   HF    voice    receivers   were    tested,    operating 
in    upper   side    band    (USB)  ,    find    4    were    found    in    need   of   main- 
tenance.     Twenty-one    UHF   voice    receivers   were    tested   and    12 
"fair"    or    "poor"    performing   units   were    found.      Eighteen    UHF 
transmitters   were    tested   and    6   were    operating  within 
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SMT  TTY  TEST 


End  equipment  tests  (TTY '  s  and  Page  Printers)  were  per- 
formed 11  and  12  December  19  72.   Eight  of  the  best  operating 
machines  were  selected  as  reference  standards  for  all  other 
HF  record  circuit  tests. 

All  types  of  TTY  distortion  were  introduced  to  each 
machine  (eight  at  a  time)  in  5%  increments,  until  the  stress 
point  of  each  machine  was  reached.   A  machine  is  considered 
to  be  in  excellent  operational  condition  if  it  can  accept  40% 
distortion  and  print  clean  copy;  35%  distortion  input  with 
clean  copy  is  considered  good;  30%  fair;  25%  marginal  and  20% 
or  below  to  be  operationally  in  poor  condition. 

Results  of  these  tests  were: 


Machine 

\ 

LTP-C 

KYBD 

LTP-C 

RCV 

LTP-B 

KYBD 

LTP-B 

RCV 

LTP-PE 

KYBD 

LTP-N4 

PP 

LTP-PF 

KYBD 

LTP-N3 

PP 

LTP-PD 

KYBD 

LTP-A 

KYBD 

LTP-A 

RCV 

LTP-N1 

PP 

*LTP-PG 

KYBD 

*LTP-N2 

PP 

LTP-N5 

PP 

LTP-N7 

PP 

LTP-TC1  PP 

LTP-PB 

RCV 

*LTP-PC 

KYBD 

LTP-PC 

RCV 

LTP-PB 

KYBD 

Max.  Distortion 
with  Clean  Copy 

40% 

35% 

40% 

No  test 

45% 

40% 

40% 

35% 

35% 

30% 

35% 

40% 

35% 

35% 

35% 

35% 

30% 

40% 

35% 

45% 

35% 


Max.  Stress 
Point 

45% 
40% 
45% 
Machine  would  not 
45% 
45% 
45% 
40% 
40% 
35% 
40% 
45% 
40% 
40% 
40% 
40% 
35% 
45% 
40% 

40% 


Type    Distortion 
at  Max.    Stress    Pt. 

Space   Bias 

Space   Bias 

Space    Bias 

space 

1   hit   on   Mark   Bias 

Space    Bias 

Mark   End 

Mark   End 

Mark   Bias 

Mark   Bias 

Mark    Bias 

All 

Space   Bias 

Mark    End 

Space    Bias 

Space    Bias 

Mark    Bias 

Space    Bias 

Mark    End 

1    hit    on    Space    End 

Mark    Bias 


Metro    CH    #8 


25% 


30% 


Mark    Bias 


*NOTE 


LTP-PG      KYBD      Margin    adjustment    required    -    3 
LTP-N2      PP  Spacing    and    line    feed   problem 

LTP-PC      KYBD      Margin    adjustment    required    -    2 

[38] 


characters  short 
characters  short 


Table  (5 


!) 
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specifications .   Seven  HF  transmitters  were  tested  and  none 
were  found  to  be  operating  within  specifications.   [38] 

E.   PROBLEMS  WITH  PRESENT  PROGRAM  AND  AREAS  FOR  FURTHER  STUDY 

1.  The  SMT  program  has  completed  some  system  and  sub- 
system feasibility  studies  and  needs  to  devote  additional 
attention  to  its  final  two  levels  of  testing,  i.e.,  equipment 
and  circuits. 

2.  Once  laboratory  procedures  and  instrumentation  re- 
quirements have  been  established  alternate,  suitable,  test 
equipment  must  be  examined  and  a  trade-off  analysis  conducted 
between  equipment  cost  and  equipment  capability  in  order  to 
reduce  the  price  of  each  set  of  test  equipment  from  $4  8,000 
to  the  stated  goal  of  $5,000  per  set,  while  still  satisfying 
the  mission  of  SMT.   The  ideal  cost  effective  solution,  and 
one  which  certainly  is  feasible  is  to  determine  what  pieces' 
of  test  equipment,  presently  in  the  Naval  inventory,  could  be 
used  to  perform  the  SMT  test  and  thereby  not  incur  any  addi- 
tional  equipment  expense.   The  goal  should  be:   that  by 


providing  a  trained  radioman  the  tapes ,  he  could  conduct  the 
test  at  no  additional  expense  to  the  Navy  other  than  the  cost 
of  his  training  and  the  tapes. 

3.   If  the  fleet  is  to  be  the  ultimate  user  of  SMT,  the 
fleet  must  have  trained  personnel  available  which  implies  a 
training  program,,  syllabus  and  instructors,  etc.   These  needs 
must  be  planned  for  as  long  lead  time  items  so  they  can  be 
available  when  SMT  is  vailable  for  fleet  use. 
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4.  One  of  the  goals  of  the  SMT  program  is  to  replace 
several  less  efficient  tests  presently  being  used.   These 
tests  and/or  procedures  which  are  no  longer  needed  must  be 
identified  and  removed  from  the  inventory.   The  SMT  program 
should  be  able  to  pay  for  itself  with  tests ,  equipment  and 
training  it  replaces  while  at  the  same  time  it  enhances  the 
tests  which  it  complements. 

5.  The  SMT  program  must  keep  in  mind  its  long  range 
goals  and  specific  objectives  and  continue  to  devote  its 
attention  in  these  areas .   Work  with  the  fleet  should  be 
secondary  to  these  goals  and  objectives  until  the  total 
package  is  prepared.   By  dividing  its  attention  between  pro- 
gram development  and  fleet  applications  it  will  most  likely 
result  in  a  less  efficient  product  and  it  will  extend  the 
development  phase  longer  than  would  be  otherwise  necessary. 
These  detractions  must  be  weighed  against  possible  benefits 
of  present  fleet  utilization. 

6.  When  Naval  Electronic  Systems  Command  has  completed 
the  SMT  development  program,  it  must  have  prepared  an  imple- 
mentation plan  or  OPORDER  to  introduce  its  technology  to  the 
fleet.   The  transfer  of  technology,  to  be  effective  and  fully 
utilized,  must  be  prepared,  programmed  and  conducted  in  a 
well  planned  and  executed  maneuver. 

7.  Once  SMT  testing  procedures  for  communications 
System,  Sub-system,  equipment  and  circuits  have  been  designed, 
an  adequate  level  of  testing  at  each  level  must  be  developed. 
The  depth,  intensity  and  frequency  of  tests  for  components 
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and  systems  must  be  determined.   The  areas  of  degraded  mode 
(total  systems  degradation  or  component  failure  which  re- 
quires use  of  an  alternate  procedure)  and  systems  sensitivity 
to  atmospheric  or  operational  phenomena  must  be  determined. 

8.   Studies  must  be  conducted  to  determine  the  time  period 
a  calibrated  (SMT  tested)  system  remains  in  calibration.   In 
order  to  ensure  optimum  systems  availability  should  the  sys- 
tem be  tested  daily,  weekly,  once  a  quarter,  etc.?   Once 
this  "validity  interval"  is  known  an  implementation  program 
must  be  drawn  up  which  would  select  either  or  both  of  the 
following  courses  of  action: 

1)  If  the  validity  interval  is  determined  to  be  measur- 
able in  extended  periods  of  time  (month  or  more)  the 
SMT  test  sequence  could  be  administered  to  a  ship  as 
part  of  its  regular  overhaul,  pre-deployment  ORI ,  or 
during  every  in-port  period.  This  approach  would 
dictate  mobile  teams  stationed  in  key  domestic  and 
foreign  ports. 

2)  If  the  validity  interval  is  measurable  in  hours,  days 
or  weeks  it  may  be  desirable  to  have  each  major 
operating  unit  able  to  test  itself  on  a  frequent 
schedule.   This  approach  would  permit  a  ship  to  con- 
tinuously monitor  its  communication  system  and  achieve 
maximum  availability.   This  solution  would  be  expen- 
sive in  terms  of  manpower,  equipment  and  training. 

A  trade-off  analysis  should  be  conducted  between  the 
resultant  availability  and  cost. 
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It  may  be  determined  that  the  optimum  solution  is  a  blend 
of  the  two  approaches  with  major  ships  carrying  its  own  SMT 
equipment  while  mobile  teams  provide  periodic  readiness  in- 
spections.  Once  either  or  both  alternatives  is   selected, 
implementation  plans  must  be  prepared  and  funding  sought  to 
accomplish  the  established  goal. 

9.   Voice  measuring  systems  must  be  refined.   The  present 
procedure  calls  for  looking  at  three  100  Hz  windows  in  the 
voice  band  from  200  to  6100  Hz  and  correlating  the  data  to 
determine  intelligibility.   Studies  must  be  continued  to 
determine  the  optimum  location  of  the  three  windows  and  to 
refine  the  procedure. 

10.  Studies  must  be  conducted  to  determine  the  signal 
reproduction  loss  due  to  multipath  reception  and  the  effect 
of  quad  diversity  upon  the  UCC-1. 

11.  Cross  talk  between  systems  and  components  must  be 
studied  so  that  its  effects  can  be  predicted,  tested  and 

corrected. 

12.  Studies  of  compatibility  of  the  SMT  procedures  and 
Secure  systems  must  be  conducted.   Until  Secure  systems  can 
be  incorporated  into  communications  systems  test  sequence  the 
sequence  will  not  be  able  to  realize  its  full  potential. 

13.  The  validity  of  the  atmospheric  test  must  be  in- 
creased.  This  can  be  accomplished  by  improving  the  atmo- 
spheric simulation  techniques. 
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VI.   POTENTIAL  BENEFITS  AND  APPLICATIONS  OF  SMT 

The  potential  applications  and  benefits  to  be  derived 
from  applications  of  the  SMT  concept  are  numerous.   The  ben- 
efits include  increased  standardization,  more  definitive 
specifications  for  procurement,  reduction  of  communication 
systems  duplication,  reduced  manpower  needed  to  support 
communications,  increased  availability  and  a  means  to  mea- 
sure the  degraded  level  at  which  a  system  is  operating. 
Numerous  potential  applications  are  also  listed. 

A.   POTENTIAL  BENEFITS  OF  SMT 

Standardization  is  one  of  the  basic  principles  upon  which 
the  SMT  concept  was  founded  and  it  is  the  key  to  the  high 
degree  of  success  that  has  been  achieved  during  the  initial 
phase  of  development.   The  standardization  principles  employed 
in  the  development  of  the  SMT  concept  should  also  be  applied 
to  the  development  of  new  equipment  by  Navy  laboratories  and 
commercial  contractors  to  ensure  that  the  same  techniques  are 
used  to  establish  performance  standards.   Standards  of  per- 
formance, based  on  SMT  requirements,  should  be  an  integral 
part  of  a  system,  sub-system,  equipment  and  component  de- 
sign specifications.   All  equipment  procurred ,  as  a  result  of 
a  production  contract,  should  comply  with  the  performance 
criteria  established  by  SMT  as  the  minimum  acceptable  for  use 
in  an  operational  environment. 
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As  a  result  of  the  use  of  SMT  procedures  in  evaluating 
systems  performance,  standardization  of  test  procedures  would 
be  universal  regardless  of  the  system  configuration  or  site 
location.   Any  manager,  operator,  or  supplier  should  be  aware 
of  the  benefits  of  standardization. 

When  the  operators  and  industry  engineers  select  a  common 
unit  of  measurement,  as  the  EDR,  RLMG  and  SMT  procedures  can 
provide  for  communications,  everyone  will  speak  the  same 
language.   Definitive  specifications  can  be  stated,  and  under- 
stood and  construction  can  be  monitored  in  terms  of  quantified 
parameters,  not  qualified  criteria  as  is  often  the  practice  at 
present.   The  operator  can  plan  on  achieving  and  holding  a 
performance  level,  which  will  permit  him  to  make  meaningful 
judgments  as  to  his  needs  and  capabilities. 

Once  a  common  unit  of  measurement  has  been  selected  all 
communications  systems  can  be  compared  and  contrasted  with 
each  other.   It  is  quite  certain  that  duplication  of  communi- 
cation systems  exists  between  members  of  the  Armed  Forces , 
numerous  other  government  agencies,  and  within  industry. 
Duplication  is  taken  to  mean  two  systems  which  perform  the 
same  task  or  similar  tasks  and  for  which  research  and  develop- 
ment costs ,  learning  curve  costs  and  implementation  costs 
were  duplicated.   If  all  concerned  agencies  had  used  one  mea- 
surement scale  it  would  have  been  possible  to  prevent  the 
needless  additional  expense  of  developing  and/or  building 
and/or  introducing  the  same  system  twice. 
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With  a  test  procedure  such  as  SMT,  communications  systems 
manpower  requirements  could  be  reduced  in  several  ways. 
Since  an  entire  external  communications  system  can  be  checked 
in  5%  to  20%  of  the  time  needed  by  present  test  procedures, 
less  personnel  are  needed  to  perform  the  tests.   Less  down 
time  is  needed  to  perform  maintenance.   Systems  and  components 
can  be  tested  while  installed  which  means  equipment  does  not 
have  to  be  removed  and  replaced  and  it  is  not  exposed  to 
damage  while  handling.   Additionally,  only  defective  com- 
ponents need  be  removed  and  a  better  indication  of  the  failure 
can  be  provided  greatly  easing  the  maintenance  requirement 
which  means  less  personnel  are  needed  to  perform  overhaul  or 
depot  level  maintenance. 

The  SMT  testing  procedures  will  enable  the  user  to  deter- 
mine the  level  of  performance  of  his  system.   The  user  will 
be  aware  of  any  degradation  of  the  system's  performance.   He 
will  be  able  to  measure  the  level  of  degradation  and  make  a 
decision,  based  upon  quantitative  information,  regarding  his 
ability  to  complete  an  assigned  mission. 

Proper  utilization  of  the  SMT  techniques  will  increase 
the  communications  systems  availability.   Since  availability 
can  be  achieved  through  maintainability  and  maintainability 
needs  an  adequate  test  procedure  and  SMT  is,  not  only,  an 
adequate  test  procedure ,  but  an  improvement  over  any  other 
available  test  procedure,  system  availability  will  increase 
which  means  the  system  operator  and  user  will  derive  a  benefit 
from  utilization  of  the  Standard  Measurement  Technique.   Since 
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SMT  measures  degraded  mode  operations ,  maintenance  can  be 
planned  in  anticipation  of  a  catastrophic  failure  which  means 
the  system  should  not,  as  often,  suddenly  be  put  out  of 
commission.   When  catastrophic  failure  does  occur,  SMT  will 
permit  quicker  trouble  shooting  and  permit  a  faster  restoration 
of  the  failed  system  or  component. 

Numerous  additional  benefits  may  be  derived  by  implemen- 
tation of  SMT.   When  a  garbled  message  is  received,  the 
receiver  could  test  his  system  to  ensure  it  is  operating 
within  specifications.   He  could  inform  the  sender  of  the 
garbled  message  and  the  sender  could  do  likewise.   It  may 
even  be  advantageous  to  have  the  SMT  test  signal  broadcast 
from  one  unit  to  another  in  order  to  test  the  communications 
system  on  a  macro  level. 

An  individual  unit  could  introduce  a  series  of  messages 
into  its  own  receiver  system  and  then  observe  how  the  messages 
are  handled  by  its  local  message  distribution  system. 

Taking  this  idea  one  step  further  could  enable  SMT  to 
make  its  greatest  contribution  to  Naval  Communications.   Since 
SMT  can  measure  the  level  of  performance  of  a  communications 
system,  as  on  a  ship,  and  assign  a  quantitative  value  to  it, 
it  could  assign  a  quantitative  value  to  a  total  ship's 
communications  suit  by  testing  all  on-board  communications 
systems  and  summing  their  level  of  performance.   A  method  of 
weighting  the  various  systems  would  have  to  be  devised.   In 
this  manner  it  could  be  stated  that  ship  X  can  communicate 
with  a;  Y  (30%),  level  of  distortion  with  a  Z;  (95%)  ,  confidence 
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interval.   Once  this  is  done  every  ship  in  the  Navy  could  have 
a  quantitative  value  (s)  assigned  to  its  communications  suit. 
Weak  performers  could  receive  more  attention  and  strong  per- 
formers checked  to  see  what  beneficial  practices  could  be 
utilized  by  other  vessels.   The  same  concept  could  be  applied 
to  all  Naval  Communications  Stations  whereby  their  ability  to 
communicate  could  be  quantified.   Once  ships  and  stations 
levels  of  performance  have  been  determined  the  test  could  be 
applied  from  the  communications  stations  to  the  ship  and  vice 
versa  by  transmitting  the  test  tape.   This  step  would  enable 
the  performance  level  of  the  trunk  to  be  quantified. 

Up  to  this  point  we  have  addressed  equipment  orientated 
applications.   It  is  possible  to  include  personnel,  message 
handling,  distribution  and  management.   By  using  test  tapes 
with  a  pre -determined  sequence  of  messages  introduced  at  the 
communications  station,  the  flag  ship,  or  at  an  individual 
ship's  receiver,  the  ability  of  a  communications  system  to 
put  a  message  in  the  hands  of  the  ultimate,  intended  user 
could  be  evaluated.   This  tape  could  include  messages  dealing 
with  operations,  administration,  supply,  etc.,  and  range  from 
FLASH  to  ROUTINE  in  priority.   In  this  manner,  all  aspects 
of  the  total  communications  system,  i.e.,  equipment,  personnel, 
management,  etc.,  could  be  tested  and  weak  points  identified. 
The  test  could  be  repeated  at  any  location  or  unit  and  permit 
a  valid,  equal  comparison  between  two  or  more  communi cat ions 
systems . 
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Test  tapes  could  be  transmitted  via  satellite  in  order 
to  determine  the  level  of  performance  at  which  various 
satellite  channels  are  operating. 

Since  there  is  no  major  communications  system  test  avail- 
able/ SMT  would  complement  present  sub-system  and  component 
tests-   Some  tests  will  be  redundant  and  no  longer  needed  and 
they  can  be  removed  from  the  inventory  producing  a  cost 
savings.   However  the  two  primary  benefits  of  SMT  must  not  be 
overshadowed.   First  it  is  a  means  to  check  a  communications 
system,  not  components,  and  secondly  it  provides  a  means  to 
measure  degraded  performance.   Since  very  few  systems  are 
operating  at  full  performance  levels,  an  ability  to  measure 
the  degraded  level  of  performance  is  necessary. 

Retrofit  installation  of  a  system  and/or  equipment  is 
an  expensive  process.   The  early  acceptance  of  SMT  as  a 
standard  installation  for  new  construction  could  result  in  a 
considerable  savings.   As  an  interim  measure,  consideration 
should  be  given  to  the  installation  of  sensors  required  for 
SMT  in  new  construction  pending  a  decision  to  include  SMT  in 
initial  outfitting. 

B.   POTENTIAL  APPLICATIONS  OF  SMT 

The   potential    application   of   Standard  Measurement   Tech- 
niques   (SMT)    to   electronic    systems,    other   than    afloat/ashore 
communication    systems,    is   based   on    the    documented   successful 
achievement   of   initial    objectives    obtained    in    the    development 
of   SMT    as    an    integral   part   of    the   Navy's    overall    System 
Performance    Measurement   Program. 
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The  application  of  the  SMT  engineering  concept  to  the 
communication  links  of  Tactical  Data  Systems  such  as  NTDS , 
ATDS,  and  MTDS ,  is  a  natural  extension  of  the  initial  appli- 
cation of  SMT.   Testing  hardware  in  TDS  equipped  aircraft 
(E2C,  P3C,  F14)  using  real  world,  simulated  inputs,  and  em- 
ploying SMT  procedures  for  determining  the  performance  of 
data  links  prior  to  being  airborne,  would  be  valuable.   The 
SMT  concept  could  also  be  employed  while  airborne  to  monitor 
data  link  performance,  determine  source  of  data  link  problems 
(own  aircraft  or  termination  station) ,  and  isolate  equipment 
that  is  operating  below  established  performance  levels . 

The  techniques  of  SMT  are  universal  in  application  to 
electronic  systems.  Although,  the  initial  application  is 
for  testing  naval  communication  systems,  SMT  is  applicable  to 

a.  Tactical  Data  Systems 

b.  Avionic  Systems 

c.  ASW  Systems 

d.  Weapons  Systems 

e.  Interior  Communication  Systems  (ICS) 

f.  Command  and  Control  Systems 

g.  Navigation  Systems 

h.   Aircraft  Voice  Systems 

i.   Submarine  Communications  Systems 

j .   ECM  Systems 

Plans  have  been  drawn  up  to  "Hard  Wire"  the  SMT  system 

into  the  communications  system  of  the  USS  NIMITZ  (CVAN-6  8) 

[23]  and  into  the  Patrol  Frigate  (PF)  class  which  is  in  the 

development  stage  [39]. 
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VII.   CONCLUSION 

The  need  for  a  Naval  Communications  System  Level  Test 
Procedure  exists.   Until  this  need  is  satisfied,  personnel 
operating  the  system  will  not  know  or  be  able  to  use  all  the 
potential  of  the  system  which  they  are  working  with.   Once 
the  level  of  performance  of  a  system  can  be  quantified,  using 
meaningful  criteria,  such  as,  the  amount  of  degradation  which 
it  can  accept  and  still  be  usable  for  communications  or  what 
percentage  of  the  system  is  operating  above  a  predetermined 
level,  the  managers  of  the  system  can  determine  how  much 
through-put  the  system  can  provide.   This  knowledge,  capa- 
bility and  availability,  is  the  first  step  towards  efficient 
utilization  of  present  and  future  resources.   A  test  pro- 
cedure is  needed  which  can  provide  this  knowledge. 

It  has  been  shown  that  all  existing  test  procedures  are 
inadequate  and  mislabeled.   Virtually  all  are  called  systems 
level  test  procedures  when  they  are  really  subsystem  level 
procedures  measuring' output  power,  percentage  modulation, 
etc. 

A  system  level  test  procedure  should  be  able  to  accomplish 
the  following: 

1)  Quantify  the  level  of  performance  of  the  system  or 
determine  the  present  level  of  degradation  at  which  the  sys- 
tem is  operating. 

2)  Determine  how  much  degradation  in  the  input  signal  a 
system  can  accept  and  still  put  out  an  error-free  or  acceptable 
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3)   Determine  the  communications  system  ability  to 
communicate;  its  reason  for  being. 

The  Standard  Measurement  Technique  (SMT) ,  presently  under 
development  by  the  Naval  Electronic  Systems  Command,  Southwest 
Division,  is  a  system  level  test  which  can  fulfill  this  need. 
By  inputing  signals  of  known  degradation  and  quantifying  the 
output  it  can  measure  the  ability  of  a  system  to  pass  infor- 
mation.  However,  SMT  is  not  an  end-all.   It  does  not  provide 
all  the  information  which  a  manager  needs  in  order  to  operate 
his  system.   It  should  be  used  to  compliment  some  present  sub- 
systems level  procedures  while  others  should  be  deemphasized. 
SMT  can  be  an  excellent  tool  in  the  hands  of  the  operators 
and  when  used  in  conjunction  with  QMS  can  provide  a  complete, 
instantaneous  systems  status  report.   It  can  be  used  to 
point  out  where  the  maintenance  effort  should  be  applied. 
PMS/3M  and  Pomsee  should  be  applied  as  maintenance  procedures 
to  failed  components  located  by  SMT  and  QMS. 

SMT  development,  by  Naval  Electronic  Systems  Command, 
must  be  completed.   Alternate  cost  effective  pieces  of  test 
equipment  must  be  found.   Test  procedures  must  be  firmed  and 
alternative  applications  must  be  examined.   The  SMT  validity 
interval  needs  to  be  determined  so  that  a  decision  can  be  made 
which  will  evaluate  the  costs  and  benefits  of  applying  SMT 
to  operational  units  on  a  periodic  schedule,  such  as  once  a 
quarter  or  before  a  cruse,  or  having  it  built  into  all  units 
so  it  can  be  used  continuously. 
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The  Standard  Measurement  Technique  has  the  potential  to 
deliver  system  level  testing  and  provide  Naval  Communications 
with  a  complete,  true,  system,  integrated  test  structure. 
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APPENDIX  A 
Major  Test  Equipment  Functions 

The  functions  of  the  major  test  equipment  used  in  the 
'P"  system  test  are  as  follows: 

a.  Magnetic  tape  deck  and  interface  buffer  used  to  in- 
put prerecorded  standard  test  message  tapes. 

b.  Spectrum  Analyzer  is  used  to  measure  the  individual 
channel  levels  of  a  composite  tone,  i.e.,  distortion 
levels,  inter-modulation. 

c.  Dummy  load  to  absorb  the  full  power  transmit  signal 
to  acceptable  testing  levels. 

d.  Attenuators  used  to  reduce  the  receiver  input  voltage 
levels  to  acceptable  testing  levels . 

e.  Micro-voltmeter   used   to   measure    voltage    level    at 
receiver. 
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